Human-computer Interaction Research Research Articles

Using generative AI in human resource development: an applied research study

ABSTRACT In this article, we report results of an experimental study investigating the use of ChatGPT in the design of an HRD intervention. Our goal was to identify the benefits and drawbacks of using this technology in HRD practice. The theoretical framework for the study was provided by the Activity Theory, an often used approach in Human-Computer Interaction research. The study participants were HRD doctoral students from two large public universities in the USA. The participants used ChatGPT in developing an HRD intervention plan, based on a supplied organisational scenario. Data for the analysis included the intervention plans, prompts used by participants, outputs received from ChatGPT, and participants’ responses to open-ended questions. The study revealed varied participant experiences with ChatGPT, including differences in time investment and interaction rules. Among key findings were the tendency to interact with ChatGPT as if it were a human consultant, and steps to establish division of labour between the tool and the human agent. ChatGPT was valued for its usefulness in initial planning and brainstorming ideas for HRD interventions, yet its limitations were apparent in generic responses and lack of contextual understanding. Across all participants, the crucial role of human expertise in refining ChatGPT’s output was consistently highlighted.

Towards more trusted virtual physicians: the combinative effects of healthcare chatbot design cues and threat perception on health information trust

ABSTRACT Although virtual healthcare assistants have been extensively adopted in the healthcare industry, users still question their reliability. Based on the Computers are Social Actor (CASA) paradigm, the current study conducted an experiment to examine how different chatbot design cues (Chatbot vs. Layperson vs. Doctor) affect users’ trust towards virtual healthcare assistants. Results indicate that the doctor-like design cues and bot-like design cues (vs. layperson-like design cues) elicited significantly greater perceived chatbot expertise, which enhanced users’ trust in health information. This study further found the significant moderating effects of users’ perceived threat on chatbot expertise and privacy concerns. Positive effects of the doctor-like design cue and the bot-like design cue on perceived expertise were found to be significant only for those whose perceived threat was high. Interestingly, the doctor-like design cues led to greater (less) privacy concerns when the perceived threat level was low (high). The findings provide human–computer interaction researchers and chatbot UX/UI designers with important theoretical and practical implications.

Towards identifying the components of students' AI literacy: An exploratory study based on Hungarian higher education students' perceptions

With the advent of the popular use of artificial intelligence (AI), the higher education (HE) sector is now facing a new challenge regarding how to exploit the educational potentials of Human-computer interaction (HCI). Developing students’ AI literacy is now attracting the attention of the HE and HCI research community. This paper aims to explore HE students’ perceptions of efficient and critical use of AI tools, and to systematically map the potential components of AI literacy as a new 21st century skill for HE students. This study applied a qualitative, exploratory approach in the form of semi-structured interviews with HE students. Results indicate that the participants primarily use ChatGPT for tasks such as brainstorming, topic selection, searching for information, and translation. While many find it useful for creating and reformatting texts, some encountered challenges, including generality in responses, outdated information, and issues during exams. Students highlighted its effectiveness in various academic tasks, from writing essays and CVs to language learning and transcription. Instructors’ perspectives on ChatGPT varied, with some advocating for its integration, while others expressed concerns about job security and misinformation. The implications of the study call for a more systematic introduction and discussion around AI literacy in educational settings. 

Examining the Integration of a GPS-based Staff Performance Monitoring and Assessment System within Academic Settings

Research evidence of manipulations of appraisal systems of various institutions in Nigeria abound, and little is known about the application of a Geographical positioning system (GPS) enabled system in this context. To cover this gap, this study investigates the integration of a real-time staff performance monitoring and assessment system with GPS technology, within academic settings. With the increasing emphasis on quality education and efficient resource utilization, academic institutions are exploring innovative approaches to monitor staff performance and student outcomes. The integration of real-time performance systems offers a promising avenue to achieve these goals by providing timely feedback, fostering accountability, and facilitating data-driven decision-making processes. This study outlines the key components of such a system, including its technological infrastructure, data collection methods, and performance evaluation metrics. Additionally, it explores the potential benefits and challenges associated with its adoption, considering factors such as staff engagement, privacy concerns, and institutional culture. Through a comprehensive examination of existing literature and case studies, this research aims to provide valuable insights and recommendations for academic institutions seeking to leverage real-time staff performance monitoring for institutional enhancement and educational excellence. While this study highlights the significance of stakeholder collaboration and continuous improvement mechanisms in ensuring the successful integration and sustained effectiveness of the monitoring system, it introduces a nuanced novel GPS technology and highlights some design implications which will guide future human-computer interaction researchers.

HCI Research and Innovation in China: A 10-Year Perspective

In the past years, human computer interaction (HCI) research and innovation have developed substantially, leading to a number of fruitful research topics. In this paper, we surveyed the HCI research and innovation in China from a 10-year perspective. We analyzed the popular research methodology and topics among Chinese researchers, including human modeling, user interface techniques, context awareness, user acceptance and performance, user experience design, human-AI interaction, HCI applications and social influences. We also conducted a bibliography analysis on the published papers in top-tier conferences and journals, which revealed a significant rising trend, and a generally broad distribution of research types. Moreover, we described typical applications and the industry influence of the research outcomes. We concluded with implications and reflections for HCI researchers across the world and shared the future research trends envisioned by Chinese researchers.

Human-Computer Interaction in Smart Living Environments: Challenges and Opportunities

The proliferation of smart technologies in living environments has ushered in an era where humans and computers interact more intimately than ever before. This research paper delves into the complex landscape of Human-Computer Interaction (HCI) within the context of smart living environments, shedding light on the multifaceted challenges and exciting opportunities that lie ahead.
 In this exploration, we first review the fundamental concepts and technologies that underpin smart living environments, emphasizing the centrality of HCI in bridging the gap between humans and the myriad of interconnected devices and systems that constitute these environments. We then identify and analyze key challenges such as user privacy, security, usability, and adaptability, which continue to shape the HCI landscape in smart living.
 The paper also highlights the emerging opportunities for HCI research, including the integration of augmented reality (AR), virtual reality (VR), and natural language processing (NLP) into smart living systems, fostering more intuitive and seamless interactions. Additionally, it discusses the potential for personalized and context-aware HCI solutions that enhance user experiences and promote inclusivity.
 By addressing these challenges and harnessing these opportunities, this research seeks to provide valuable insights for HCI researchers, designers, and policymakers, ultimately contributing to the development of smarter and more user-friendly living environments that enhance our quality of life while respecting our fundamental rights and values.

Bibliometric Analysis of the Research Landscape in Human-Computer Interaction in Ibero-America

Human-Computer Interaction (HCI) is a research discipline that optimizes human-machine interfaces. Ibero-America has become a leading region in HCI advances adapted to its unique socioeconomic environment. However, a global characterization of the research landscape for this discipline in the region still needs to be improved. Therefore, this study aimed to analyze the scientific literature on HCI in Ibero-America using bibliometric techniques, identifying key authors, institutions, conceptual structure, and research trends. The methodology employed consisted of a bibliometric analysis of 9812 documents from 1990-2023 from Scopus and Web of Science. The data were analyzed using Bibliometrix and VOSviewer for statistical and network analysis. The results revealed that Spain, Brazil, Portugal, Mexico, and Colombia are at the forefront of HCI research in the region, with more than 90 % of total publications experiencing an annual growth rate of approximately 14.6 %, driven by strong international collaborations. Key topics are human-computer interaction, virtual reality, augmented reality, and video games, while education is one of the main application areas. The main conclusion is that the HCI research landscape in Ibero-America is characterized by rapid evolution, international integration, and conceptual diversity, indicative of a mature and interdisciplinary field that responds to regional demands. The advancement of the field could be driven by increased private-sector funding. Future system designs in the region should prioritize cultural sensitivity and inclusion.

Capturing the College Experience

Understanding the dynamics of mental health among undergraduate students across the college years is of critical importance, particularly during a global pandemic. In our study, we track two cohorts of first-year students at Dartmouth College for four years, both on and off campus, creating the longest longitudinal mobile sensing study to date. Using passive sensor data, surveys, and interviews, we capture changing behaviors before, during, and after the COVID-19 pandemic subsides. Our findings reveal the pandemic's impact on students' mental health, gender based behavioral differences, impact of changing living conditions and evidence of persistent behavioral patterns as the pandemic subsides. We observe that while some behaviors return to normal, others remain elevated. Tracking over 200 undergraduate students from high school to graduation, our study provides invaluable insights into changing behaviors, resilience and mental health in college life. Conducting a long-term study with frequent phone OS updates poses significant challenges for mobile sensing apps, data completeness and compliance. Our results offer new insights for Human-Computer Interaction researchers, educators and administrators regarding college life pressures. We also detail the public release of the de-identified College Experience Study dataset used in this paper and discuss a number of open research questions that could be studied using the public dataset.

Human-Computer Interaction in Agricultural User Interfaces

This study explores the intersection of Human-Computer Interaction (HCI) and agricultural user interfaces, aiming to enhance the efficiency and user experience within the agricultural domain. A thorough literature review identifies current trends, challenges, and gaps in HCI research specific to agricultural interfaces. The research objectives encompass the proposal of HCI design principles tailored to the unique characteristics of the agricultural environment. Methodologically, the study employs user surveys, interviews, and usability testing to gather insights into user preferences and challenges. The paper presents case studies highlighting successful implementations of HCI principles in agricultural settings, providing tangible examples of improved user engagement and task performance. Challenges and limitations in HCI adoption in agriculture are discussed, along with user feedback and evaluations shaping interface design. The implications of the research extend to HCI practitioners, agricultural technology developers, and policymakers, offering practical insights for future developments. This paper contributes valuable knowledge to the evolving field of HCI in agricultural contexts, emphasizing the importance of user-centered design.
  

Voice-based age, gender, and language recognition based on ResNet deep model and transfer learning in spectro-temporal domain

In personal identity recognition systems, detecting a person's age, gender, and language using voice signal characteristics is a crucial issue, especially because of the importance of security considerations. Age, gender, and language classification problems are important in signal processing because they are used to analyze and understand human behavior, interactions, and preferences. This can be especially useful in the fields of human-computer interaction, psychology, and social science research. In this paper, a new system for detecting a speaker's age, gender, and language based on deep learning models is presented. Deep learning models have shown great efficacy in various fields of signal processing. For this paper, a range of deep models were tested, including convolutional neural networks (CNNs), recurrent neural network (RNN), and a fine-tuning ResNet34 architecture. Additionally, techniques such as transfer learning were applied to improve the efficiency of the proposed system. The input voice signals are preprocessed by applying the spectro-temporal transform to obtain additional features that can be fed to the ResNet34 model, which is designed specifically for the task of voice signal processing. The dataset used in this paper was sourced from the Mozilla common voice initiative, which is dedicated to advancing speech recognition and language identification technologies. The performance of the proposed algorithm was evaluated in the presence of Gaussian noise to determine its robustness. The experimental results demonstrated that the proposed algorithm significantly outperformed basic algorithms and other deep neural networks in terms of age and gender recognition from voice signals.

Coral Morph: An Artistic Shape-Changing Textile Installation for Mindful Emotion Regulation in the Wild

Mindful human–computer interaction has been increasingly used to induce emotion regulation. The Coral Morph is an artistic shape-changing textile installation designed for mindful emotion regulation in public spaces. The installation incorporates affective soft-robotic material movements, heart rate physicalization, and expressive interactive textile features conceived to enhance sensory engagement. In an exhibition, 55 participants were recruited to interact with Coral Morph, allowing the evaluation of its sensory engagement, somesthetic appreciation, and lifelikeness perception. Participants’ facial expressions during interaction were triangulated with their questionnaire responses, and short interviews were conducted enquiring on the somatic feedback after the interaction. The empirical findings confirmed that participants perceived the installation as emotionally intelligent, animate, likable, safe, and interesting. Participants regarded the experience as pleasant, calm, positive, and relaxed, appreciating the somesthetic aspects, thus highlighting the effectiveness of the design elements. Furthermore, we highlighted the significance of individual differences in shaping the interactive encounter. These findings inform well-being and human–computer interaction researchers on designing future responsive installations for mindfulness and identifying future research avenues, encompassing the nuanced exploration of interactive material behaviors and advancements in evaluation methodologies.

Emotion Detection via Voice and Speech Recognition

In human-computer interaction, spoken emotion detection is essential yet difficult. Researchers extract emotional information from speech sounds using a variety of analytic techniques. Effective communication requires an understanding of human emotions, which range from happiness to astonishment and beyond. Classification, pre-processing, feature extraction, and pattern recognition are all involved in emotion recognition. While some approaches take into account a continuous spectrum, others concentrate on a single-dimensional representation. The study of emotional cognition benefits from the use of technologies such as fMRI, which facilitate the distinction between positive and negative emotions. The intricacy and significance of identifying emotions in speech are emphasised in this abstract, underscoring the necessity of sophisticated methodologies in the field of human-computer interaction research

I Know This Looks Bad, But I Can Explain: Understanding When AI Should Explain Actions In Human-AI Teams

Explanation of artificial intelligence (AI) decision-making has become an important research area in human–computer interaction (HCI) and computer-supported teamwork research. While plenty of research has investigated AI explanations with an intent to improve AI transparency and human trust in AI, how AI explanations function in teaming environments remains unclear. Given that a major benefit of AI giving explanations is to increase human trust understanding how AI explanations impact human trust is crucial to effective human-AI teamwork. An online experiment was conducted with 156 participants to explore this question by examining how a teammate’s explanations impact the perceived trust of the teammate and the effectiveness of the team and how these impacts vary based on whether the teammate is a human or an AI. This study shows that explanations facilitate trust in AI teammates when explaining why AI disobeyed humans’ orders but hindered trust when explaining why an AI lied to humans. In addition, participants’ personal characteristics (e.g., their gender and the individual’s ethical framework) impacted their perceptions of AI teammates both directly and indirectly in different scenarios. Our study contributes to interactive intelligent systems and HCI by shedding light on how an AI teammate’s actions and corresponding explanations are perceived by humans while identifying factors that impact trust and perceived effectiveness. This work provides an initial understanding of AI explanations in human-AI teams, which can be used for future research to build upon in exploring AI explanation implementation in collaborative environments.

Understanding the Role of Human-Computer Interaction (HCI) in User friendliness in Navigating the Computer System

Abstract: This research paper aims to provide an in-depth exploration of the field of HCI, tracing its historical development, examining its core concepts, methods, and approaches, and shedding light on its profound impact on society. Furthermore, it will delve into emerging trends in HCI, the challenges it faces, and the opportunities it offers. The paper will conclude by discussing the future directions of HCI research and its continued relevance in a rapidly evolving technological landscape.

Picturing the fictitious person: An exploratory study on the effect of images on user perceptions of AI-generated personas

Human-computer interaction (HCI) research is facing a vital question of the effectiveness of personas generated using artificial intelligence (AI). Addressing this question, this research explores user perceptions of AI-generated personas for textual content (GPT-4) and two image generation models (DALL-E and Midjourney). We evaluate whether the inclusion of images in AI-generated personas impacts user perception or if AI text descriptions alone suffice to create good personas. Recruiting 216 participants, we compare three AI-generated personas without images and those with either DALL-E or Midjourney-created images. Contrary to expectations from persona literature, the presence of images in AI-generated personas did not significantly impact user perceptions. Rather, the participants generally perceived AI-generated personas to be of good quality regardless of the inclusion of images. These findings suggest that textual content, i.e., the persona narrative, is the primary driver of user perceptions in AI-generated personas. Our findings contribute to the ongoing AI-HCI discourse and provide recommendations for designing AI-generated personas.

Modelling conversational agent with empathy mechanism

Empathy mechanism in communication is the cornerstone for effective and meaningful interaction. Establishing an empathy mechanism in conversation agent (CA) requires accurate recognition of users’ emotions to facilitate generates appropriate empathetic responses. Therefore, we proposed a Multimodal Emotion Recognition Model (MERM) to recognizes a user’s emotional state from multimodal data (audio, facial expressions, and conversation text) during conversation, and an Interactive Empathetic Conversation Model (IECM) to generate empathetic responses based on the MERM. Comparative and ablation study results indicated that the proposed models outperform existing methods in recognizing the user’s emotions and generating appropriate empathetic responses. We also conducted an experiment study, the results indicated that the CA significantly enhances the user’s emotional experience.

TMBL: Transformer-based multimodal binding learning model for multimodal sentiment analysis

Multimodal emotion analysis is an important endeavor in human–computer interaction research, as it enables the accurate identification of an individual’s emotional state by simultaneously analyzing text, video, and sound features. Although current emotion recognition algorithms have performed well using multimodal fusion strategies, two key challenges remain. The first challenge is the efficient extraction of modality-invariant and modality-specific features prior to fusion, which requires deep feature interactions between the different modalities. The second challenge concerns the ability to distinguish high-level semantic relations between modality features. To address these issues, we propose a new modality-binding learning framework and redesign the internal structure of the transformer model. Our proposed modality binding learning model addresses the first challenge by incorporating bimodal and trimodal binding mechanisms. These mechanisms handle modality-specific and modality-invariant features, respectively, and facilitate cross-modality interactions. Furthermore, we enhance feature interactions by introducing fine-grained convolution modules in the feedforward and attention layers of the transformer structure. To address the second issue, we introduce CLS and PE feature vectors for modality-invariant and modality-specific features, respectively. We use similarity loss and dissimilarity loss to support model convergence. Experiments on the widely used MOSI and MOSEI datasets show that our proposed method outperforms state-of-the-art multimodal sentiment classification approaches, confirming its effectiveness and superiority. The source code can be found at https://github.com/JackAILab/TMBL.

Comparative Study of K-NN, Naive Bayes and SVM for Face Expression Classification Techniques

Abstract Classification is a data mining technique used to predict group membership for data instances within a given dataset. Classification is considered as an example of supervised learning as training data associated with class labels is given as input. This comparative study focuses on the application of K-Nearest Neighbors (K-NN), Naive Bayes, and Support Vector Machines (SVM) classification techniques in the context of face expression recognition. The study aims to evaluate and compare the performance of these techniques in accurately classifying facial expressions, their advantages and disadvantages. Facial expression recognition is an important area in computer vision and emotion analysis. It involves identifying and classifying different facial expressions such as happiness, sadness, anger, fear, surprise and disgust. Accurate classification of facial expressions is crucial for various applications, including human-computer interaction, affective computing and psychological research. K-NN is a non-parametric algorithm that classifies data based on the majority class of its K-nearest neighbors. Naive Bayes is a probabilistic algorithm that makes classification decisions based on the Bayes theorem and assumes independence between features. SVM is a machine learning algorithm that constructs a hyperplane to separate different classes of data. Dataset such as Affectnet is used to conduct the study including labeled facial expression images. Three algorithms are tested on this dataset to evaluate their performance in correctly classifying facial expressions. Performance metrics such as accuracy, precision, recall, and F1-score are used to assess the classification performance of each technique. Software used for conducting the experiments is Python.

Blueprints: Systematizing Behavior Change Designs—The Case of Social Comparison Theory

To improve people’s lives, human-computer interaction researchers are increasingly designing technological solutions based on behavior change theory, such as social comparison theory (SCT). However, how researchers operationalize such a theory as a design remains largely unclear. One way to clarify this methodological step is to clearly state which functional elements of a design are aimed at operationalizing a specific behavior change theory construct to evaluate if such aims were successful. In this article, we investigate how the operationalization of functional elements of theories and designs can be more easily conveyed. First, we present a scoping review of the literature to determine the state of operationalizations of SCT as behavior change designs. Second, we introduce a new tool to facilitate the operationalization process. We term the tool blueprints . A blueprint explicates essential functional elements of a behavior change theory by describing it in relation to necessary and sufficient building blocks incorporated in a design. We describe the process of developing a blueprint for SCT. Last, we illustrate how the blueprint can be used during the design refinement and reflection process.

Patient Acceptance of Self-Monitoring on a Smartwatch in a Routine Digital Therapy: A Mixed-Methods Study

Self-monitoring of mood and lifestyle habits is the cornerstone of many therapies, but it is still hindered by persistent issues including inaccurate records, gaps in the monitoring, patient burden, and perceived stigma. Smartwatches have the potential to deliver enhanced self-reports, but their acceptance in clinical mental health settings is unexplored and rendered difficult by a complex theoretical landscape and need for a longitudinal perspective. We present the Mood Monitor smartwatch application for mood and lifestyle habits self-monitoring. We investigated patient acceptance of the app within a routine 8-week digital therapy. We recruited 35 patients of the UK’s National Health Service and evaluated their acceptance through three online questionnaires and a post-study interview. We assessed the clinical feasibility of the Mood Monitor by comparing clinical, usage, and acceptance metrics obtained from the 35 patients with a smartwatch with those from an additional 34 patients without a smartwatch (digital treatment as usual). Findings showed that the smartwatch app was highly accepted by patients, revealed which factors facilitated and impeded this acceptance, and supported clinical feasibility. We provide guidelines for the design of self-monitoring on a smartwatch and reflect on the conduct of human-computer interaction research evaluating user acceptance of mental health technologies.